Long-term Preservation of Earth Observation Data and Knowledge in ESA through CASPAR

ESA-ESRIN, the European Space Agency Centre for Earth Observation (EO), is the largest European EO data provider and operates as the reference European centre for EO payload data exploitation. EO Space Missions provide global coverage of the Earth across both space and time generating on a routine continuous basis huge amounts of data (from a variety of sensors) that need to be acquired, processed, elaborated, appraised and archived by dedicated systems. Long-term Preservation of these data and of the ability to discover, access and process them is a fundamental issue and a major challenge at programmatic, technological and operational levels. Moreover these data are essential for scientists needing broad series of data covering long time periods and from many sources. They are used for many types of investigations including ones of international importance such as the study of the Global Change and the Global Monitoring for Environment and Security (GMES) Program. Therefore it is of primary importance not only to guarantee easy accessibility of historical data but also to ensure users are able to understand and use them; in fact data interpretation can be even more complicated given the fact that scientists may not have (or may not have access to) the right knowledge to interpret these data correctly. To satisfy these requirements, the European Space Agency (ESA), in addition to other internal initiatives, is participating in several EU-funded projects such as CASPAR (Cultural, Artistic, and Scientific knowledge for Preservation, Access and Retrieval), which is building a framework to support the end-to-end preservation lifecycle for digital information, based on the OAIS reference model, with a strong focus on the preservation of the knowledge associated with data. In the CASPAR Project ESA plays the role of both user and infrastructure provider for one of the scientific testbeds, putting into effect dedicated scenarios with the aim of validating the CASPAR solutions in the Earth Science domain. The other testbeds are in the domains of Cultural Heritage and of Contemporary Performing Arts; together they provide a severe test of preservation tools and techniques. In the context of the current ESA overall strategies carried out in collaboration with European EO data owners/providers, entities and institutions which have the objective of guaranteeing long-term preservation of EO data and knowledge, this paper will focus on the ESA participation and contribution to the CASPAR Project, describing in detail the implementation of the ESA scientific testbed. 1 This paper is based on the paper given by the authors at the 5th International Digital Curation Conference, December 2009; received November 2009, published December 2009. The International Journal of Digital Curation is an international journal committed to scholarly excellence and dedicated to the advancement of digital curation across a wide range of sectors. ISSN: 1746-8256 The IJDC is published by UKOLN at the University of Bath and is a publication of the Digital Curation Centre. 5 Sergio Albani & David Giaretta Introduction ESA-ESRIN, the European Space Agency establishment in Frascati (Italy), is the largest European Earth Observation (EO) data provider and operates as the reference European centre for EO payload data exploitation. EO data provide global coverage of the Earth across both a continuum of timescales and a variety of geographical scales. EO data are generated by many different instruments producing multi-sensor data in long time series with a variable geographical coverage, variable geometrical resolution and variable temporal resolution. EO data acquired from space therefore constitute a powerful scientific tool to support better understanding and management of the Earth and its resources. More specifically, large international initiatives such as ESA-EU GMES (Global Monitoring for Environment and Security) and the intergovernmental GEO (Group on Earth Observations) focus on coordinating international efforts in environmental monitoring, that is, to provide political and technical solutions to global issues, such as climate change, global environment monitoring, management of natural resources and humanitarian response. At present several thousand ESA users worldwide (Earth scientists, researchers, environmentalists, climatologists, etc.) have online access to EO missions’ metadata (millions of references), data (in the range of 3 to 5 PB) and derived information for long-term science and long-term environmental monitoring; moreover the requirements for accessing historical archives have greatly increased over the last years and the trend is likely to keep increasing. Therefore, the prospect of losing the digital records of science (and in particular the unique data, information and publications managed by ESA) is very alarming. Issues for the near future include: • the identification of the type and amount of data to be preserved; • the location of archives and their replication for security reasons; • the detailed technical choices (e.g. formats, media); • the availability of adequate funds. Of course decisions should be taken in coordination with other data owners together with the support and advice of the user community. ESA Overall Strategies Currently the major constraints are that the data volumes are increasing dramatically (ESA plans of new missions indicate 5-10 times more data to be archived in the next 10-15 years). The available financial budgets are inadequate (preservation and access to data of each ESA mission are covered only until 10 years after the end of the mission). Futhermore, data preservation/access policies are different for each EO mission and each operator or Agency. To respond to the urgent need for a coordinated and coherent approach for the long-term preservation of the existing European EO space data, ESA started consultations with its member States in 2006 in order to develop the European LTDP (Long Term Data Preservation) strategy, presented at DOSTAG (Data, Operations, Scientific and Technical Advisory Group) in 2007, and also formed a LTDP Working 2 European Space Agency (ESA) Portal http://www.esa.int/ The International Journal of Digital Curation Issue 3, Volume 4 | 2009 Long-term Preservation of Earth Observation Data 6 Group (January 2008) within the GSCB (Ground Segment Coordination Body) to define European LTDP Common Guidelines (in cooperation with the European EO data stakeholders) and to promote them in CEOS (Committee on Earth Observation Satellites) and GEO (Group on Earth Observations). This group is defining an overall strategy for the long-term preservation of all European EO data, ensuring accessibility and usability for an unlimited time-span, through a cooperative and harmonized collective approach among the EO data owners (European LTDP Framework) by the application of European LTDP Common Guidelines. ESA member states, as part of ESA's mandatory activities, have currently approved a three-year initial LTDP programme with the aim of establishing a full long-term data preservation concept and programme by 2011. ESA is now starting the application of the European LTDP Common Guidelines to its own missions. Highpriority ESA LTDP activities for the next 3 years are focused on issues such as security improvement, migration to new technologies, increase in the number of datasets to be preserved and enhancement of data access. In addition ESA-ESRIN is participating in a number of international projects partially funded by the European Commission and concerned with technology development and integration in the areas of long-term data preservation and distributed data processing and archiving. The scope of ESA participation to such LTDP related projects is: • to evaluate new technical solutions and procedures to maintain leadership in using emerging services in EO; • to share knowledge with other entities, also outside the scientific domain; • to extend the results and outputs of these cooperative projects in other EO (and ESA) communities.